Distance detection by global shutter image sensor

ABSTRACT

A method of detecting a distance, comprising emitting an optical pulse, receiving at a first detector a reflected optical pulse, from a first exposure start time to a first exposure finish time, generating a first detector signal, receiving at an n-th detector the reflected optical pulse, from an n-th exposure start time to an n-th exposure finish time, generating an n-th detector signal, wherein the first exposure start time begins before the n-th exposure start time and the first exposure finish time ends before the n-th exposure finish time and the first exposure duration partially overlaps the n-th exposure duration and determining the distance of an object based on the first detector signal and the n-th detector signal.

BACKGROUND

The instant disclosure relates to method of measuring a distance usingoffset shutters on an image sensor.

Global shutter image sensors may be utilized to detect the motion ofobjects since the pixels in an image sensor can collect lightsimultaneously. This advantage can be used to detect very short pulsesof infrared light reflected from an object in order to detect a distancebetween device and the object. The speed of light is such that in theautomotive market a gated complementary metal oxide semiconductor (CMOS)sensor has only tens of picoseconds of exposure time to capture thereflected infrared (LR) light that may be used. This picosecond exposuretime complicates the sensor designs and production and limits thesensitivity due to the short photon capture times.

Therefore, a more cost efficient and effective method of distancedetection has been disclosed.

SUMMARY

In one embodiment a method of detecting a distance, comprising emittingan optical pulse, the optical pulse having a pulse beginning time and apulse ending time, receiving at a first detector a reflected opticalpulse, wherein the optical pulse is detected at the first detector froma first exposure start time to a first exposure finish time and has afirst exposure duration, generating a first detector signal based on afirst response of the first detector to the reflected optical pulse,receiving at an n-th detector the reflected optical pulse, wherein theoptical pulse is detected at the n-th detector from an n-th exposurestart time to an n-th exposure finish time and has an n-th exposureduration, generating an n-th detector signal based on an n-th responseof the n-th detector to the reflected optical pulse, wherein the firstexposure start time begins before the n-th exposure start time and thefirst exposure finish time ends before the n-th exposure finish time andthe first exposure duration partially overlaps the n-th exposureduration and determining the distance of an object based on the firstdetector signal and the n-th detector signal.

In another embodiment a method of detecting a distance, comprisingemitting an optical pulse, the optical pulse having a pulse beginningtime and a pulse ending time, receiving at a first detector a reflectedoptical pulse, wherein the optical pulse is detected at the firstdetector from a first exposure start time to a first exposure finishtime and has a first exposure duration, generating a first detectorsignal based on a first response of the first detector to the reflectedoptical pulse, receiving at an m-th detector the reflected opticalpulse, wherein the optical pulse is detected at the m-th detector froman m-th exposure start time to an m-th exposure finish time and has anm-th exposure duration, generating an m-th detector signal based on anm-th response of the m-th detector to the reflected optical pulse,wherein the first exposure start time begins after the m-th exposurestart time and the first exposure finish time ends before the m-thexposure finish time and the first exposure duration is within the m-thexposure duration and determining the distance of an object based on thefirst detector signal and the m-th detector signal.

In yet another embodiment a method of detecting a distance, comprisingemitting an optical pulse, the optical pulse having a pulse beginningtime and a pulse ending time, receiving at a first detector a reflectedoptical pulse, wherein the optical pulse is detected at the firstdetector from a first exposure start time to a first exposure finishtime and has a first exposure duration, generating a first detectorsignal based on a first response of the first detector to the reflectedoptical pulse, receiving at an n-th detector the reflected opticalpulse, wherein the optical pulse is detected at the n-th detector froman n-th exposure start time to an n-th exposure finish time and has ann-th exposure duration, generating an n-th detector signal based on ann-th response of the n-th detector to the reflected optical pulse,receiving at an m-th detector the reflected optical pulse, wherein theoptical pulse is detected at the m-th detector from an m-th exposurestart time to an m-th exposure finish time and has an m-th exposureduration, generating an m-th detector signal based on an m-th responseof the m-th detector to the reflected optical pulse, wherein the n-thexposure start time begins after the first exposure start time and them-th exposure start time begins before the first exposure start time andthe n-th exposure finish time and the m-th exposure finish time endsafter the first exposure finish time and the first exposure durationpartially overlaps the n-th exposure duration and the first exposureduration is within the m-th exposure duration and determining thedistance of an object based on the first detector signal, the n-thdetector signal and the m-th detector signal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a depiction of an optical pulse reflection and single shuttersamplings.

FIG. 2 is a depiction of an overlapping sequential sampling of areflected optical pulse according to example embodiments.

FIG. 3 is a depiction of an overlapping nested sampling of a reflectedoptical pulse according to example embodiments.

FIG. 4 is a depiction of an overlapping hybrid sampling of a reflectedoptical pulse according to example embodiments.

FIG. 5 is a depiction of a first and an n-th discrete sensor accordingto example embodiments.

FIG. 6 is a depiction of a first and an m-th discrete sensor accordingto example embodiments.

FIG. 7 is a depiction of a first, an n-th and an m-th discrete sensoraccording to example embodiments.

FIG. 8 is a depiction of a first and an n-th composite sensor accordingto example embodiments.

FIG. 9 is a depiction of a first and an m-th composite sensor accordingto example embodiments.

FIG. 10 is a depiction of a first, an n-th and an m-th composite sensoraccording to example embodiments.

DETAILED DESCRIPTION

It may be readily understood that the components of the presentapplication, as generally described and illustrated in the figuresherein, may be arranged and designed in a wide variety of differentconfigurations. Thus, the following detailed description of the examplesof a method as represented in the attached figures, is not intended tolimit the scope of the application as claimed, but is merelyrepresentative of selected examples of the application.

The features, structures, or characteristics of the applicationdescribed throughout this specification may be combined in a suitablemanner in one or more examples. For example, the usage of the phrasesexample, examples, some examples, or other similar language, throughoutthis specification refers to the fact that a particular feature,structure, or characteristic described in connection with the examplemay be comprised in at least one example of the present application.Thus, appearances of the phrases example, examples, in some examples, inother examples, or other similar language, throughout this specificationdoes not necessarily refer to the same group of examples, and thedescribed features, structures, or characteristics may be combined in asuitable manner in one or more examples.

The reflection of an infrared beam by shuttered global image sensors maybe used to calculate a distance from the sensors regardless of a pulsepattern. For example, an infrared (IR) light source may be placed at afirst end and a global shutter image sensor placed at a second end. Thespeed of light is 299,792,458 m/s. The start of infrared light emissionand the start exposure of the global shutter sensor may be aligned forsimultaneous triggering. If the exposure duration of the global shuttersensor is 66.7 ns and the receiving pattern matches the emission patternof the light source, then the infrared beam travels has traveledapproximately 20 meters. The exposure duration is the difference betweenthe time the shutter is gated open to the time the shutter is gatedclosed. It is also to be noted that the emitted light may be patternedto distinguish it from various other IR signals.299,792,458÷10000,000×0.0667=19.996 meters

If both the light source and the image sensor are placed at the sameend, the sensor receives the light pattern reflected by an object. Thenthe calculated distance between the sensor and object is approximately10 meters.19.996÷2=9.998 meters

If a car is assumed to be moving at a speed of 200 km/hr toward thedetector which is equipped with a global shutter sensor triggering at120 frames per second. When the detector collects one image frame of theinfrared pattern, the car has moved 0.46 meters. Therefore, this devicecan emit and collect 21 samples before this car moves more than 10meters from its original position. If both exposure time and lightsource pulse duration are kept at 66.7 ns, and the infrared light startemission is 1 us before the time when the sensor starts to collectlight, when the image matches the light source pattern, the distancebetween that car and detector is:299,792,458÷1,000,000÷2=149.896 meters

The distance is calculated from the time difference between when thelight source is emitted and the matched pattern image is captured. If afaster frame rate is utilized, more samples may be collected to enhancethe image pattern search. The shorter exposure time the sensor has, thehigher the precision. In one example sensor design with 100 MHz systemclock may be able to support 10 ns exposure times for ±3 meterprecision.

Short exposure times limit the sensitivity of the sensor, as theydecrease the signal to noise ratio (SNR) of images increases which maycause accuracy issues. The present disclosure depicts variousconfigurations which may improve the accuracy without shortening theexposure time.

FIG. 1 depicts an optical pulse that has been reflected 120 from anobject. The reflected optical pulse 120 has a rising edge 130 and afalling edge 140. An optical receiver has a shutter exposure starting attime X and finishing at time Y. In one instance the shutter exposurestart 142 begins past the reflected optical pulse falling edge andexposure end 144 also ends after the reflected optical pulse fallingedge. In another instance the shutter exposure start 146 begins beforethe pulse rising edge and exposure ends 148 after the pulse fallingedge. In yet another instance the shutter exposure begins 150 before thepulse rising edge and exposure ends 152 at the onset of the pulse risingedge. The pulse itself may be patterned so that it may be correctlydifferentiated from stray optical pulses. Also, it is evident that theconcurrence of the pulse and the shutter may not occur at all, or may bemarginal.

Using multiple sensors whose exposure starting times may be variouslytriggered is suggested. The use of an external signal to spread out thesensors' exposure time in the time domain to collect multiple imagesfrom the multiple sensors may increase the pattern samples. Overlayingthe exposure period among those sensors may enhance the precision.

In one example, two sensors are configured with the same frame rate andthe same exposure time and the second sensor starts exposure half wayafter the first sensor exposure. If the match pattern is found on firstsensor but not on second sensor, then the pulse was received in thefirst half of the first sensor exposure and the precision is improved by4×. If both sensors catch the pattern, then the pulse has beenintercepted at the overlap of the two exposures and the precision isagain improved by 4×. The location of the reflected pulse is determinedby calculating the pattern density difference between these two images.

FIG. 2 depicts the same optical pulse that has been reflected as inFIG. 1. In this example, N overlapping shutter exposures captureportions of the optical pulse 120. The first shutter exposure starts attime A 210 and finishes at time B 220. A portion of the optical pulsehas been captured by this first sensor. The n-th shutter exposure startsat time C 230 and finishes at time D 240. Another portion of the opticalpulse has been captured by this n-th sensor. If the intensity of thelight is measured for both the first and n-th sensor, the center of thepulse may be determined from a time standpoint and an approximatedistance determined. If the n-th shutter exposure duration is the sameas the first shutter exposure duration and offset by ½ of the shutterexposure duration, then the precision of the measured distance may beincreased by four times.

The more sensors that are incorporating by this method to further dividedown the exposure overlay period, the further precision may be improved.Additionally, the exposure time may be different for the two sensors.For example, the first sensor may have a 30 ns exposure time and thesecond sensor may have a 60 ns exposure time and starts exposure 10 nsahead and ends the exposure 20 ns after the first pulse. As long as theimages are determined to occur during non-overlay exposure period, theprecision may be improved by proper image processing of the densitydifference.

FIG. 3 depicts the same optical pulse that has been reflected as inFIG. 1. In this example, M nested shutter exposure captures portions ofthe optical pulse 120. The first shutter exposure begins at time A 210and ends at time B 220. A portion of the optical pulse has been capturedby this first sensor. The m-th shutter exposure begins at a time E 310that precedes time A 210 and extends past the B 220 to time F 320. Inthis way the intensity of the first sensor and the m-th sensor may beutilized to determine a distance of the object.

FIG. 4 depicts a hybrid approach, the same optical pulse that has beenreflected as in FIG. 1 is referenced. The first shutter exposure beginsat time A 210 and ends at time B 220. The optical pulse has beencaptured by this first sensor. The n-th shutter exposure begins at timeC 230 and ends at time D 240. Another portion of the optical pulse hasbeen captured by this n-th sensor. The m-th shutter exposure begins at atime E 310 that precedes time A 210 and extends past the B 220 to time F320. In this example, both the offset and nested exposure may beutilized to determine a distance of the object.

FIG. 5 depicts a two sensor example, a first sensor 510 and an N-thsensor 520 are arranged in a planar configuration. This sensorconfiguration may also be utilized for the example shown in FIG. 2. Thesensors may be coplanar to measure extended distances or in anon-coplanar orientation to distinguish near field objects.

FIG. 6 depicts another two sensor example, a first sensor 510 and anm-th sensor 610 are arranged in a planar configuration. This sensorconfiguration may also be utilized for the example shown in FIG. 3. Thesensors may be coplanar to measure extended distances or in anon-coplanar orientation to distinguish near field objects.

FIG. 7 depicts a three sensor example, a first sensor 510, an n-thsensor 520 and an m-th sensor 610 are arranged in a planarconfiguration. This sensor configuration may also be utilized for theexample shown in FIG. 4. The sensors may be coplanar to measure extendeddistances or in a non-coplanar orientation to distinguish near fieldobjects.

The multiple sensor configuration methods described may be used insingle sensor if the sensor has multiple exposure control circuits, suchas high dynamic range (HDR) sensors. HDR sensors may divide its array toseveral sub-arrays and each sub-array having exposure control circuitsto spread out the overlay exposure period.

FIG. 8 depicts a first example of a single sensor with multiple subarrays, the first sensor is 810 and offset in X and/or Y to the n-thsensors 820. This sensor configuration may also be utilized for theexample shown in FIG. 2.

FIG. 9 depicts a second example of a single sensor with multiple subarrays, the first sensor is 810 and offset in X and/or Y to the m-thsensors 910. This sensor configuration may also be utilized for theexample shown in FIG. 3.

FIG. 10 depicts a third example of a single sensor with multiple subarrays, the first sensor is 810 and offset in X and/or Y to the n-thsensors 820 and m-th sensors 910. This sensor configuration may also beutilized for the example shown in FIG. 4.

View angle differences among those sensors may also be used to detectdepth information to map a near field distance. A suitable applicationof this example is when the object is close to the detector and due toframe rate limitation, multiple pattern samples may not be able to becollected. In this situation, using a view angle difference may increasethe accuracy of distance measurement.

There are several types of sensors on current market which have bothvisible light and IR light sensing capability pixels within one sensor,such as red, blue, green, infrared (RGBIR) sensor. Both visible lightinformation and IR light information may be captured in the same imageframe. This type of sensor may be used to calculate view angledifferences in both visible light and IR light.

Therefore, using single or multiple global shutter sensors having bothvisible light and IR light sensing capabilities may be used to detectdistance. Within the same captured images, the view angle differences inboth visible and IR images may be used to calculate the distance of nearfield objects. These sensors' exposure time may be arranged to have anoverlay period. The IR light travel time difference may be used todetect far field objects' distance and visible light information of farobjects may be referenced in the IR image pattern search.

Although exemplary examples the method of the present disclosure havebeen illustrated in the accompanied drawings and described in theforegoing detailed description, it will be understood that theapplication is not limited to the examples disclosed, and is capable ofnumerous rearrangements, modifications, and substitutions withoutdeparting from the spirit or scope of the disclosure as set forth anddefined by the following claims.

The above examples are for illustrative purposes and are not intended tolimit the scope of the disclosure or the adaptation of the featuresdescribed herein to particular components. Those skilled in the art willalso appreciate that various adaptations and modifications of theabove-described preferred examples may be configured without departingfrom the scope and spirit of the disclosure. Therefore, it is to beunderstood that, within the scope of the appended claims, the disclosuremay be practiced by examples in addition to those specificallydescribed.

What is claimed is:
 1. A method of detecting a distance, comprising:emitting an optical pulse, the optical pulse having a pulse beginningtime and a pulse ending, time; receiving at a first detector a reflectedoptical pulse, wherein the optical pulse is detected at the firstdetector from a first exposure start time to a first exposure finishtime and has a first exposure duration; generating a first detectorsignal based on a first response of the first detector to the reflectedoptical pulse; receiving an n-th detector the reflected optical pulse,wherein the optical pulse is detected at the n-th detector from an n-thexposure start time to an n-th exposure finish time and has an n-thexposure duration; generating an n-th detector signal based on an n-thresponse of the n-th detector to the reflected optical pulse; receivingat an m-th detector the reflected optical pulse, wherein the opticalpulse is detected at the m-th detector from an m-th exposure start timeto an m-th exposure finish time and has an m-th exposure duration;generating an m-th detector signal based on an n-th response of the n-thdetector to the reflected optical pulse; wherein the n-th exposure starttime begins after the first exposure start time and the m-th exposurestart time begins before the first exposure start time and the n-thexposure finish time and the m-th exposure finish time end after thefirst exposure finish time and the first exposure duration partiallyoverlaps the n-th exposure duration and the first exposure duration iswithin the m-th exposure duration; and determining the distance of anobject based on the first detector signal, the n-th detector signal andthe m-th detector signal.
 2. The method of claim 1 wherein thedetermining of distance is based on intensity of the first detectorsignal, intensity of the n-th detector signal and intensity of the m-thdetector signal.